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RELATED APPEALS AND INTERFERENCES 

There are no related appeals and interferences. 

STATUS OF THE CLAIMS 

This is an appeal from the Final Rejection dated July 13, 2004 of claims 1-2, 4-7, 9-18, 
and 28-33. No other claims are pending. 

STATUS OF AMENDMENTS 

All Amendments filed in this application have been entered. A correct copy of appealed 
claims 1-2, 4-7, 9-18, and 28-33, including all entered amendments thereto, appears in the 
attached Appendix. 

SUMMARY OF THE INVENTION 

The invention is directed to a content-based multimedia system and method. The system 
according to one embodiment includes a first color quantizer which extracts a color histogram of 
query multimedia data, a second color quantizer which extracts a color histogram of multimedia 
data to be retrieved, and a histogram converter which converts the color histogram of one of the 
extracted query multimedia data and the multimedia data to be retrieved into a histogram having a 
color space and color quantization method of the other of the extracted query multimedia data 
and the multimedia data to be retrieved. For example, referring to the embodiment shown in 
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Figure 3 of the present application and the corresponding specification disclosure, the disclosed 
system includes a first color quantizer 308, a second color quantizer 308, and a histogram 
converter 303. 

The present invention is also directed to a content-based multimedia method. The 
method according to one embodiment includes inputting query multimedia data, converting a 
color histogram of one of the input query multimedia data and multimedia data to be retrieved 
into a color histogram having a color space and color quantization method of the other of the 
input query multimedia data and the multimedia data to be retrieved so as to be the same as each 
other, and calculating a similarity between the query multimedia data and multimedia data to be 
retrieved on the basis of the converted color histogram and outputting a retrieval result in 
accordance with the calculated similarity. For example, referring to the embodiment shown in 
Fig. 5 of the present application and the corresponding specification disclosure, a query 
multimedia data, for example, in this case a query image, is input at step S500. After a color 
histogram for the query image is extracted in steps S501-S505, the color histogram of the query 
image is converted into a color histogram having a color space and color quantization method of 
another input query image. Then, in step S507 (~S508), a similarity is calculated between the 
query image and image to be retrieved on the basis of the converted color histogram. 

The invention is further directed to a content-based multimedia retrieval method for 
retrieving multimedia data by comparing query multimedia data with multimedia data to be 
retrieved. The method according to one embodiment of the invention includes extracting a 



Serial No.: 09/785,443 Docket No.: P-0187 

color histogram of the query multimedia data, extracting a color histogram of the multimedia 
data to be retrieved, comparing the extracted color space and color quantization method of the 
query image with the color space and color quantization method of the multimedia data to be 
retrieved, and converting the color histogram of one of the extracted query multimedia data and 
the multimedia data to be retrieved into a color histogram having a same color space and color 
quantization method as the other of the extracted query multimedia data and the multimedia 
data to be retrieved, when the color spaces and color quantization methods of the extracted 
multimedia data and multimedia data to be retrieved are different each other, and performing a 
retrieval in accordance with a similarity between the query multimedia data and multimedia data 
to be retrieved. For example, referring to the embodiment shown in Figure 5 of the present 
application and the corresponding specification disclosure, the method of Figure 5 includes 
extracting a color histogram of a query multimedia data, in this embodiment, a query image, in 
step S503, and reading or extracting a color histogram of a feature database about multimedia 
data to be retrieved, in this embodiment a retrieval object image, in step S504. Then, the 
method compares the extracted color space and color quantization method of the query image 
with the color space and color quantization method of the multimedia data to be retrieved, in 
step S505, and converts the color histogram of one of the extracted query multimedia data and 
the multimedia data to be retrieved into a color histogram having the same color space and color 
quantization method as the other of the extracted query multimedia data and the multimedia 
data to be retrieved in step S506. Step S509 is the retrieval step. 
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Additionally, the invention is directed to a content-based multimedia retrieval method 
according to another embodiment. The method according to another embodiment includes 
comparing the color spaces and color quantization methods of the query multimedia data and 
multimedia data to be retrieved, converting the color histogram of the query multimedia data or 
color histogram of the multimedia data to be retrieved when the color space and color 
quantization method of the query multimedia data and the color space and color quantization 
method of the multimedia to be retrieved are different, and calculating a similarity between the 
converted or unconverted query multimedia data and multimedia data to be retrieved, and 
performing a retrieval in accordance with the calculated similarity. For example, referring to the 
embodiment shown in Figure 5 of the present application and the corresponding specification 
disclosure, the method compares the extracted color space and color quantization method of the 
query multimedia data, in this embodiment, a query image, with the color space and color 
quantization method of the multimedia data to be retrieved, in step S505, and converts the color 
histogram of one of the extracted query multimedia data and the multimedia data to be retrieved 
into a color histogram having the same color space and color quantization method as the other if 
the extracted query multimedia data and the multimedia data to be retrieved in step S506. In 
step S507, a similarity is calculated between the query image and the image to be retrieved on the 
basis of the converted color histogram. Step S509 is the retrieval step. 

The invention is further directed to a content-based multimedia retrieval system 
according to another embodiment The system according to another embodiment includes a 
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first color quantizer which extracts a color histogram of multimedia data to be retrieved, a 
description means for describing color space and color quantization method of an extracted 
color histogram, and a second color quantizer which extracts a color histogram of query 
multimedia data using a method which is same as the described color space and color 
quantization method in order to perform the multimedia data retrieval Referring, for example, 
to the embodiment shown in Fig. 3 of the present application and the corresponding 
specification disclosure, the system includes a color quantizer 308, description means 306, 307, 
and a color quantizer 300. 

The invention is also directed to a content-based multimedia retrieval method according 
to another embodiment. The method according to another embodiment includes judging 
whether a color histogram of query multimedia data corresponding to a color space and 
quantization method of multimedia data to be retrieved is stored in advance, and calculating a 
similarity between the color histogram of the stored query multimedia data and the color 
histogram of the multimedia data to be retrieved and performing a multimedia retrieval in 
accordance with the calculated similarity. Referring, for example, to the embodiment shown in 
Fig. 5 of the present application and the corresponding specification disclosure, the method 
shown in Fig. 5 includes judging whether there is a color histogram of the query image which is 
extracted before (S501). If yes, the method reads a color histogram value of the query image 
extracted before (S502), reads a color histogram value of a feature database about a retrieval 
object image (S504), determines whether the color histogram of the query image is different 
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from the color histogram (color space and color quantization method) of the image to be 
retrieved (S505), converts the color histogram value of the query image (S506), and calculates 
similarity by comparing the color histogram of the query image with the color image to be 
retrieved (S507). The method then determines whether the similarity is larger than a certain 
threshold value (S508), and if yes, outputs the image to be retrieved as a similar image (S509). 

ISSUES 

1. Whether claims 1-2, 4-7, 9-11, 13-18 and 28-33 are properly rejected under 35 
U.S.C. §102(e) as being anticipated by Bergman et al. (hereinafter "Bergman"), U.S. Patent No. 
5,564,263. 

2. Whether claim 12 is properly rejected under 35 U.S.C. §103(a) as being 
unpatentable over Bergman in view of Yaung, U.S. Patent No. 6,512,850. 

GROUPING OF THE CLAIMS 
Appealed claims 1-2 and 5 form a single group and stand or fall together. Claim 1 is the 
only independent claim in this group. 

Appealed claim 4 forms a single group. 

Appealed claims 6-7 and 1 1 form a single group and stand or fall together. Claim 6 is the 
only independent claim in this group. 

Appealed claims 9-10 form a single group and stand or fall together. 

Appealed claims 13-16 form a single group and stand or fall together. Claim 13 is the 
only independent claim in this group. 
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Appealed claims 17-18 form a single group and stand or fall together. Claim 17 is the 
only independent claim in this group. 

Appealed claim 28 forms a single group. 

Appealed claims 29-30 form a single group and stand or fall together. 
Appealed claims 32-33 form a single group and stand or fall together. Claim 32 is the 
only independent claim in this group. 

THE ARGUMENT 

1. 35 U.S.C. §102(e^ 

A. Claims 1. 2. and 5 

Independent claim 1 recites "a histogram converter which converts the color histogram 
of one of the extracted query multimedia data and the multimedia data to be retrieved into a 
histogram having a color space and color quantization method of the other of the color 
histogram of the extracted query multimedia data and the multimedia data to be retrieved" 
(emphasis added). Bergman does not disclose such a histogram converter. 

Rather, Bergman discloses a system for performing a histogram conversion. As shown in 
Figure 19, the system includes a search engine 1902 which converts a query color histogram Q 
of one type into a color histogram of another type, for purposes of performing an image search 
in image archives 1-3. In performing this conversion, Bergman discloses that the query 
histogram may be in an RGB color space and that the converted histogram may be a function of 
this RGB color space, i.e., the Bergman converter only performs a histogram conversion based 
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on color space . That is, at col 13, lines 52-54, Bergman states that "[ajssuming the new color 
space is derived from the RGB color space, then my hist may be obtained via a, trans formation, 
F, of rgbhist. . . " Further, it is noted that in the transformation example discussed at col 1 3, line 
57-col. 14, line 2 and referred to by the Examiner, Bergman does not disclose that search engine 
1902 performs a histogram conversion based also on color quantization, as recited in 
independent claim 1. In fact, in the example discussed at col. 13, line 57-col. 14, line 2 and 
referred to by the Examiner, both histograms appear to have the same number of bins. 
Bergman never discusses color quantization methods, which involve reducing an image to an 
appropriate number of representative colors. Further, the Examiner's comments at pages 2-4 
(paragraph 5 of the Office Action) also do not address color quantization methods. 

A histogram conversion is shown in the non-limiting embodiment of Figure 4 of the 
present application. Here, a histogram conversion is performed from one color space (RGB) and 
color quantization (R: five levels, G: five levels, B: five levels) into a histogram having another 
color space (HSV) and color quantization (H: seven levels, S: seven levels, and V: seven levels). 
The Bergman search engine does not perform a transformation of this type, but rather only 
converts histograms based on color space . 

Accordingly, it is respectfully submitted that independent claim 1 defines over Bergman. 
Dependent claims 2 and 5 are allowable at least for the reasons discussed above with respect to 
independent claim 1, from which they depend, as well as for their added features. 
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B. Dependent claim 4 

Dependent claim 4 is allowable at least for the reasons discussed above with respect to 
independent claim 1, from which it depends, as well as for its added features. Further, 
dependent claim 4 recites that the content-based multimedia retrieval system further comprises a 
description means for describing color space and color quantization information, which are the 
bases of the color histograms, wherein the description means comprises a color space 
description means for describing color space constructing the color histogram, and a 
quantization description means for describing color quantization method constructing the color 
histogram. Bergman does not disclose or suggest such a description means. 

C. Claims 6-7 and 1 1 

Independent claim 6 recites a content-based multimedia retrieval method which includes 
"converting a color histogram of one of the input query multimedia data and multimedia data to 
be retrieved into a color histogram having a color space and color quantization method of the 
other of the input query multimedia data and multimedia data to be retrieved" (emphasis added). 
Bergman does not disclose or suggest this converting step. As previously discussed, Bergman 
only performs a histogram conversion based on color space, not based on color space and color 
quantization, as recited in independent claim 6. 

Thus, it is respectfully submitted that independent claim 6 defines over Bergman. 
Dependent claims 7 and 1 1 are allowable at least for the reasons discussed above with respect to 
independent claim 6, from which they depend, as well as for their added features. 

10 
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D. Dependent claims 9- 10 

Dependent claims 9-10 are allowable at least for the reasons discussed above with respect 
to independent claim 6, from which they depend, as well as for their added features. Further, 
dependent claim 9 recites that the converting process for converting into the same histogram 
comprises judging whether the color histogram of the query multimedia data has been extracted 
previously, reading the extracted color histogram value and identifying the color space and color 
quantization method, reading a color histogram value of the multimedia data to be retrieved and 
identifying the color space and color quantization method, and converting the color histograms 
into the color histograms of the same color space and color quantization method when the color 
histogram of the query multimedia data and the color histogram of the multimedia data to be 
retrieved are not the same. Dependent claim 10 recites that the content-based multimedia 
retrieval method further comprises extracting a color histogram of the input query multimedia 
data when the color histogram of the query multimedia data has not been extracted previously. 
Bergman does not disclose or suggest such features. 

E. Claims 13-16 

Independent claim 13 recites "comparing the extracted color space and color 
quantization method of the query image with the color space and color quantization method of 
the multimedia data to be retrieved" (emphasis added). Bergman does not disclose or suggest 
these features. 
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In performing a color histogram transformation, search engine 1902 of the Bergman 
system transforms a color histogram in one color space to a color histogram in another color 
space, so that a query image may be compared to an image stored in an archive. In order to 
perform this transformation, the search engine must have knowledge of the color spaces of the 
images to be transformed. However, as discussed above, Bergman does not disclose or suggest 
comparing color quantization methods used to extract the histogram of the query image and the 
histograms for the images stored in its archives during a transform. 

Independent claim 13 also recites "converting the color histogram of one of the extracted 
query multimedia data and the multimedia data to be retrieved into a color histogram having a 
same color space and color quantization method as the other of the extracted query multimedia 
data and the multimedia data to be retrieved" (emphasis added). As discussed above, Bergman 
does not disclose or suggest performing this conversion. 

Thus, it is respectfully submitted that independent claim 13 defines over Bergman. 
Dependent claims 14-16 are allowable at least for the reasons discussed above with respect to 
independent claim 13, from which they depend, as well as for their added features. 
F. Claims 17-18 

Independent claim 17 recites "comparing the color spaces and color quantization 
methods of the query multimedia data and multimedia data to be retrieved" (emphasis added), 
and then performing a histogram conversion. Bergman does not disclose or suggest comparing 
color quantization methods performed for query color histogram Q and the color histograms 
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formed for the images stored in its archives. Also, the claimed conversion is also performed 
based on color quantization method, a conversion which is not disclosed or suggested in 
Bergman. 

Accordingly, it is respectfully submitted that independent claim 17 defines over Bergman. 
Dependent claim 18 is allowable at least for the reasons discussed above with respect to 
independent claim 17, from which it depends, as well as for its added features. 

G. Independent claim 28 

Claim 28 recites "a description means for describing color space and color quantization 
method of an extracted color histogram" (emphasis added). Bergman does not disclose or 
suggest a description means of this type. As previously discussed, Bergman does not disclose 
that, when performing its color histogram transformations, search engine 1902 takes color 
quantization method into consideration. Also, claim 28 recites "a second color quantizer which 
extracts a color histogram of query multimedia data using a method which is same as the 
described color space and color quantization method " determined by the description means. 
Bergman also does not disclose or suggest these features. Thus, it is respectfully submitted that 
independent claim 28 defines over Bergman. 

H. Dependent claims 29-30 

Dependent claims 29-30 are allowable at least for the reasons discussed above with 
respect to independent claim 28, from which they depend, as well as for their added features. 
Further, claim 29 recites that the content-based multimedia retrieval system further comprises a 
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retrieval unit for calculating a similarity between the color histogram of the query multimedia 
data extracted before and the color histogram of the multimedia data to be retrieved, and 
outputting multimedia data in accordance with the calculated similarity as a retrieval result, while 
claim 30 recites that the content-based multimedia retrieval system further comprises a database 
for storing the color histogram of the extracted query multimedia data. Bergman does not 
disclose or suggest such a retrieval unit or database. 
I. Claims 32-33 

Claim 32 recites "judging whether a color histogram of query multimedia data 
corresponding to a color space and quanti2ation method of multimedia data to be retrieved is 
stored in advance" (emphasis added). Bergman does not disclose or suggest these features. That 
is, Bergman does not judge histograms based on color quantization nor does it perform such a 
judgment in advance of performing a similarity calculation between the color histograms. 

Thus, it is respectfully submitted that independent claim 32 defines over Bergman. 
Dependent claim 33 is allowable at least for the reasons discussed above with respect to 
independent claim 32, from which it depends, as well as for its added features. 

2. 35 U.S.C. §103(a) 

Claim 12 is allowable over Bergman for at least the reasons discussed above with respect 
to independent claim 6, from which it depends, as well as for its added features. Further, Yaung 
fails to overcome the deficiencies of Bergman as Yaung is merely cited for disclosing a threshold 
value used in performing a similarity comparison between multimedia data. Yaung, however, 
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does not disclose or suggest converting a color histogram of one of a query multimedia data and 
multimedia data to be retrieved into a color histogram having a color space and color 
quantization of the other of the query multimedia data and multimedia data to be retrieved. 
Accordingly, it is submitted that claim 12 is patentable over the Bergman- Yaung combination. 

CONCLUSION 

For at least the above reasons, it is respectfully submitted that the rejections of claims 1- 
2, 4-7, 9-18, and 28-33 should be withdrawn and the application deemed in condition for 
allowance. 

Respjectfully^submitted, 
FI 



P. O. Box 221200 
Chantilly, Virginia 20153-1200 
703 766-3701 an/uh 
Date: March 4, 2005 




Registration N ^40,287 
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APPENDIX 

1 . (Previously Presented) A content-based multimedia retrieval system, comprising 
a first color quantizer which extracts a color histogram of query multimedia data; 
a second color quantizer which extracts a color histogram of multimedia data to 

be retrieved; and 

a histogram converter which converts the color histogram of one of the extracted 
query multimedia data and the multimedia data to be retrieved into a histogram having a color 
space and color quantization method of the other of the extracted query multimedia data and the 
multimedia data to be retrieved. 

2. (Original) The content-based multimedia retrieval system according to claim 1, 
wherein the multimedia data are image data or video data. 

3. (Cancelled) 

4. (Previously Presented) The content-based multimedia retrieval system according 
to claim 1, wherein the content-based multimedia retrieval system further comprises a 
description means for describing color space and color quantization information, which are the 
bases of the color histograms, wherein the description means comprises: 
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a color space description means for describing color space constructing the color 
histogram; and 

a quantization description means for describing color quantization method 
constructing the color histogram. 

5. (Previously Presented) The content-based multimedia retrieval system according 
to claim 1, wherein the histogram converter converts the color histogram of the query 
multimedia data so as to be corresponding to color space and color quantization method of the 
multimedia data to be retrieved. 

6. (Previously Presented) A content-based multimedia retrieval method, comprising 
inputting query multimedia data; 

converting a color histogram of one of the input query multimedia data and 
multimedia data to be retrieved into a color histogram having a color space and color 
quantization method of the other of the input query multimedia data and the multimedia data to 
be retrieved so as to be the same as each other; and 

calculating a similarity between the query multimedia data and multimedia data to 
be retrieved on the basis of the converted color histogram and outputting a retrieval result in 
accordance with the calculated similarity. 
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7. (Original) The content-based multimedia retrieval method according to claim 6, 
wherein the multimedia data is image data or video data. 



8. (Cancelled) 

9. (Previously Presented) The content-based multimedia retrieval method according 
to claim 6, wherein the converting process for converting into the same histogram comprises: 

judging whether the color histogram of the query multimedia data has been 
extracted previously; 

reading the extracted color histogram value and identifying the color space and 
color quantization method; 

reading a color histogram value of the multimedia data to be retrieved and 
identifying the color space and color quantization method; and 

converting the color histograms into the color histograms of the same color space 
and color quantization method when the color histogram of the query multimedia data and the 
color histogram of the multimedia data to be retrieved are not the same. 

10. (Previously Presented) The content-based multimedia retrieval method according 
to claim 9, wherein the content-based multimedia retrieval method further comprises extracting 
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a color histogram of the input query multimedia data when the color histogram of the query 
multimedia data has not been extracted previously. 

11. (Original) The content-based multimedia retrieval method according to claim 6, 
wherein the process for converting into the same histogram is performed by referencing the 
color space description information and quantization description information of the multimedia 
data to be retrieved and query multimedia data. 

12. (Original) The content-based multimedia retrieval method according to claim 6, 
wherein the process for outputting the retrieval result comprises: 

comparing the calculated similarity with a certain threshold value; and 
outputting multimedia data corresponding to the color histogram of the 

multimedia data to be retrieved as a similar multimedia data when the similarity is larger than the 

certain threshold value. 

1 3 . (Previously Presented) A content-based multimedia retrieval method for retrieving 
multimedia data by comparing query multimedia data with multimedia data to be retrieved, 
comprising: 

extracting a color histogram of the query multimedia data; 
extracting a color histogram of the multimedia data to be retrieved; 
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comparing the extracted color space and color quantization method of the query 
image with the color space and color quantization method of the multimedia data to be 
retrieved; and 

converting the color histogram of one of the extracted query multimedia data and 
the multimedia data to be retrieved into a color histogram having a same color space and color 
quantization method as the other of the extracted query multimedia data and the multimedia 
data to be retrieved, when the color spaces and color quantization methods of the extracted 
multimedia data and multimedia data to be retrieved are different each other, and 

performing a retrieval in accordance with a similarity between the query 
multimedia data and multimedia data to be retrieved. 

1 4. (Original) The content-based multimedia retrieval method according to claim 1 3, 
wherein the content-based multimedia retrieval method further comprises a step of performing a 
retrieval in accordance with the similarity between the extracted query multimedia data and 
multimedia data to be retrieved when the color space and color quantization method of the 
extracted query multimedia data are same as the color space and color quantization method of 
the multimedia data to be retrieved. 

1 5. (Original) The content-based multimedia retrieval method according to claim 13, 
wherein the converting process for converting the color histogram converts the color space and 

20 



Serial No.: 09/785,443 Docket No.: P-0187 

color quantization method of the query multimedia data so as to correspond to the color space 
and color quantization method of the multimedia data to be retrieved. 

16. (Original) The content-based multimedia retrieval method according to claim 13, 
wherein the process for converting the color histogram converts the color space and color 
quantization method of the multimedia data to be retrieved so as to correspond to the color 
space and color quantization method of the query multimedia data. 

17. (Original) A content-based multimedia retrieval method, comprising 
comparing the color spaces and color quantization methods of the query 

multimedia data and multimedia data to be retrieved; 

converting the color histogram of the query multimedia data or color histogram of 
the multimedia data to be retrieved when the color space and color quantization method of the 
query multimedia data and the color space and color quantization method of the multimedia to 
be retrieved are different; and 

calculating a similarity between the converted or unconverted query multimedia 
data and multimedia data to be retrieved, and performing a retrieval in accordance with the 
calculated similarity. 
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1 8. (Original) The content-based multimedia retrieval method according to claim 1 7, 
wherein the color histogram converting process converts the color histogram of query 
multimedia data so as to correspond to the color space and color quantization method of the 
multimedia data to be retrieved. 

Claims 19-27. (Cancelled) 

28. (Previously Presented) A content-based multimedia retrieval system, comprising: 
a first color quantizer which extracts a color histogram of multimedia data to be 

retrieved; 

a description means for describing color space and color quantization method of 
an extracted color histogram; and 

a second color quantizer which extracts a color histogram of query multimedia 
data using a method which is same as the described color space and color quantization method 
in order to perform the multimedia data retrieval. 

29. (Original) The content-based multimedia retrieval system according to claim 28, 
wherein the content-based multimedia retrieval system further comprises a retrieval unit for 
calculating a similarity between the color histogram of the query multimedia data extracted 
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before and the color histogram of the multimedia data to be retrieved, and outputting 
multimedia data in accordance with the calculated similarity as a retrieval result. 

30. (Original) The content-based multimedia retrieval system according to claim 29, 
wherein the content-based multimedia retrieval system further comprises a database for storing 
the color histogram of the extracted query multimedia data. 

31 . (Original) The content-based multimedia retrieval system according to claim 29, 
wherein the description means comprises: 

a color space description means for describing color space information which is 
the basis of the color histogram of the extracted multimedia data to be retrieved; and 

a quantization description means for describing color quantization information 
which is the basis of the color histogram of the extracted multimedia data to be retrieved. 

32. (Previously Presented) A content-based multimedia retrieval method, comprising: 
judging whether a color histogram of query multimedia data corresponding to a 

color space and quantization method of multimedia data to be retrieved is stored in advance 
before; and 
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calculating a similarity between the color histogram of the stored query multimedia 
data and the color histogram of the multimedia data to be retrieved and performing a multimedia 
retrieval in accordance with the calculated similarity. 

33. (Original) The content-based multimedia retrieval method according to claim 32, 
wherein the content-based multimedia retrieval method further comprises: 

quantizing and extracting the query multimedia data with the color space and 
color quantization method of the multimedia data to be retrieved when the color histogram of 
the query multimedia data is not stored before; 

storing the color histogram of the quantized and extracted query multimedia data; 

and 

calculating a similarity between the color histogram of the extracted query 
multimedia data and the color histogram of the multimedia data to be retrieved and performing a 
multimedia retrieval in accordance with the calculated similarity. 
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